{
 "cells": [
  {
   "cell_type": "markdown",
   "source": [
    "### 提升方法 AdaBoost\n",
    "\n",
    "> 理论 《统计学习方法》第8章 提升方法\n",
    ">\n",
    "> 代码 numpy version && torch version\n",
    ">\n",
    "> Python3.7\n",
    ">\n",
    "> created 2023/02/14\n",
    ">\n",
    "> author lyz\n",
    ">\n",
    "> email 2281250383@qq.com"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "提升Boosting方法在分类问题中，通过改变训练样本的权重，学习多个分类起，并将这些分类器进行线形组合，提高分类器的性能\n",
    "\n",
    "对于分类问题，给定训练集，找一个弱分类器会简单的多。提升方法就是从弱学习算法出发，反复学习，得到一系列的弱分类器（基本分类器），然后组合这些弱分类器，构成一个强分类器。\n",
    "\n",
    "大多数的Boosting都是通过改变训练数据的概率分布（权值分配），然后针对不同的训练数据分布调用弱分类器学习一系列的弱分类器"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### AdaBoost基本思想\n",
    "\n",
    "Adaboost方法是通过改变训练样本权重来学习多个弱分类器并线性组合形成强分类器的提升算法\n",
    "\n",
    "提高被前一轮弱分类器误分类的样本权重，降低正确分类的样本权重 ---> 新一轮训练时候弱分类器会更加关注前一轮被误分类的样本\n",
    "\n",
    "得到的一系列弱分类器，AdaBoost使用加权多数表决的方法 ---> 加大分类误差率小的弱分类器权值，使得其在表决中起到较大作用"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### AdaBoost算法描述\n",
    "\n",
    "现嘉定一个二分类问题\n",
    "\n",
    "输入： 训练数据集T，弱学习算法\n",
    "输出： 最终分类器$G(x)$\n",
    "1. 初始化训练数据的权值分布$D_1$。第一次相同权值$w_{1i} = 1\\over N$\n",
    "2. 一轮训练\n",
    "    1. 对$D_m$的训练数据集学习，得到基本分类器$G_M$\n",
    "    2. 计算$G_m(x)$在训练数据集上的分类误差率\n",
    "    3. 计算$G_m(x)$的系数\n",
    "    4. 更新训练数据集的权值分布\n",
    "3. 构建基本分类器的线形组合，得到最终分类器\n",
    "\n"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.datasets import make_blobs\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import  accuracy_score\n",
    "import torch\n",
    "from torch import nn\n",
    "import torch.utils.data as data\n",
    "import random"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10,)\n",
      "(10,)\n"
     ]
    }
   ],
   "source": [
    "x_train = np.array([0,1,2,3,4,5,6,7,8,9])\n",
    "y_train = np.array([1,1,1,-1,-1,-1,1,1,1,-1])\n",
    "print(x_train.shape)\n",
    "print(y_train.shape)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(200, 2)\n",
      "(200,)\n"
     ]
    },
    {
     "data": {
      "text/plain": "<Figure size 320x320 with 1 Axes>",
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAASAAAAESCAYAAABdMQgmAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAAnYAAAJ2AHHoLmtAAAviklEQVR4nO2de3QU5f3/35vdQCKLEPCgKaigUBGaYiAeSEJIwCgSq5TLN1BjFWNUaItYb4fqKV7aShR7IVYiFReprmJqoZUjaI9+haSR0CZEpPCr3CWSgF+BIBsCySbP749xkp3dmdmZ2bnufl7n5EBmdueZmey89/lcHxdjjIEgCMICkqw+AYIgEhcSIIIgLIMEiCAIyyABIgjCMkiACIKwDI+Vg19zzTW4+uqrrTwFgiAM4uDBg/j8889lX2OpAF199dXYvHmzladAEIRBFBUVRX0NmWAEQVgGCRBBEJZBAkQQhGWQABEEYRkkQARBWAYJEEEQlkECRMQPgQBQUQHMm8f9GwhYfUZEFCzNAyII3QgEgMxM4NgxoL0d2LQJePFFoLER8HqtPjtCApoBEfGBz9crPgD377Fj3PZYoZmVYZAAEfFBbW2v+PC0twOffBLbcfmZ1dKlQFUV929mJomQTpAAEfFBbi6QmirclpoK5OTEdlwjZ1YECRARJ5SWAkOH9opQair3e2lpbMc1amZFACABIuIFr5dzOJeXc76a8nJ9HNBGzawIACRARDzh9QIPPACsX8/9q0f0y6iZFQGAwvAEIQ8/s/L5OLMrJ4cTn3gK7QcC3PXV1nIzPhOvjwSIIKLBz6weeMDqM9Efi/OnyAQjiETG4igfCRBBJDIWR/kMEaA1a9YgOzsbkydPxmeffWbEEARB6IHFUT7dBejUqVOorKxEdXU1XnvtNfz85z/XewiCIPTC4iif7k7oHTt2oKCgAMnJyRg5ciS+/vprdHd3IymJrD2CsB0WR/l0F6DTp08jLS2t53ev14szZ870bPP7/fD7/QCA5uZmvYcnCEItFkb5dBegtLQ0tLa29vweCAQwYMCAnt9LSkpQUlICQNmyHQRhOhbmxSQaugvQxIkTsWzZMgSDQTQ1NWHw4MFkfhHOIZa8GBIu1eguQIMGDcK9996LvLw8uN1uvPTSS3oPQRDGIZcXI2eiUEM0TRgyNbnvvvuwfft2/POf/8S4ceOMGIIgjEFrXgy17dAE2UYEEYrWvBhq26EJEiCCCEVrXgy17dAECRBBhKK1rxC17dAEVcMTRDha8mISoW2HAZAAEYRe2K1thwPSAkiACMIorBQAh6QFkA+IIIzA6uV8HJIWQAJEEEZgtQA4JC2ABIggjMBqAXBIWgAJEEEYgdUC4JC0ABIggjACqwXAqHXSdIaiYARhBHbIC7JbWoAINAMiCKNhzOozsC00AyIII5DKw6mp4cLyNk4ONBMSIIIwArEw/JdfAhkZQFubrZMDzYRMMIIwArEw/PnzwOnTtk8ONBMSIIIwArEwfFIS0NUl3GbD5EAzIQEiCCMQC8OnpAAul/B1NkwONBMSIIIwgtA8nDlzgIsuAs6dE0bEXC4gPd3a5MBAAKio4HKFKirMq1X7FnJCE4RR8Hk4APDuu+KvWbjQOge0DSrmaQZEEFLoNTuorQU6OyO3MwY0NMR2jrFgdcEsSIAIQhw922nk5gLJyZHbPR5r/T9WF8yCBIggxNFzdlBaClx+udAB7XIBV1xhrf/H6oJZGCBAjz32GLKzszFp0iSUl5frfXiCMAc9ZwdeL7BrF/Dcc9wsKjOT+/+uXdYmIBYXR5qGnZ3cdpPQXYDuvfdebN++HZ988gneffddHDlyRO8hCMJ49J4deL3Ao48CO3cC1dVA377APfcAK1ZwP3pHoZT4r6qqODMwFI+H224WzEDy8vJYU1OT5P4ZM2YYOTxBaOfsWcZGjmQsNZUxgPt35Ehuu57Hdbm4HyPHkDpucTG3P/xn3rzYxv8WJc+3YT6gt99+G1dddRWGDRsm2O73+1FUVISioiI0NzcbNTxBxIZR/XTCfUv8Yw/oF4VS6r+ygQ9I0wzo7NmzbOLEiRE/r7zyCmOMsY8//pjdcMMN7Ny5czErJEHEFVKzDj1nIEpnNkbN8r5FyfOtKRHR6/Wirq5OdF9DQwOWLl2KzZs3IzVcXQki0cnN5RL+wh3cPHrMQMTGEDuuDZqmuRjTt1tSVlYWzp07hyFDhgAA/vCHP+C6664TfW1RURE2b96s5/AEYW/Cs4/50DxjvW1beVNP67pi4WOEH9cklDzfupdi1NfX631IghDHASt/RhA+65gwgdve0CCcgcRSJmGDmY1SdJ8BqYFmQIRmbPItbxgVFVz2dbgZVV5u6x7PoSh5vikTmnAmNqhj0p1AgMsJyswEli2zvEzCDKgannAmcpnKDpkhCAgEgHHjgMOHpZvYx2HvIJoBEc7EDjkseuLzAU1N8uJjw4UFY4UEiHAmVi/8pzdSLTsArpDVpgsLxgoJEKEOizvo9eCQlT8VI9ey45FHOLPSqdcmA0XBCOXEe+TJSsR8QC4XMGKE9qp5i9MUKApG6Es8Rp7sgt4tO/RsqGYgJECEcmzQQS+uCW3ZsXMn93+tMxaHfFmQABHKibfIkxlY5TNzyJcFCRChnHiLPBmNlWaQQ74sSIAI5cRb5MlorDSDHPJlQZnQhDr4ta6cmG0shpGRIikzaNu23v1GRKf4a8rIAL7/fS6aNmWKLQtSSYAI56C3WCipOI9lTLG+PCkpXE/oLVuMWQxQKlXitddsJz4AjO0JHQ3qiEgoJtbufWfPMrZyJdctcOXK3t/54/E/qancdr3GDH//JZfIj6mF0GubPZuxlBR9j68RS3tCE4SuxOJPkXIGb9smHymK1Ycj5jObMiV6dEpN5Cz82v7+d+D8efnja8GgaB6ZYIQziKX6XUpILroIcLuBrq7e14ZGimIZM9x0W7Om1wTizS+xMdU2Igu/ttBrETu+FgxcQ55mQIQziCWsLCUku3dHik9opEjrmHLh99JSID29t+4rOZn7nR9T7axL7NoAIClJ/Jq0YGA0jwSIcAaxhJXFhAQQtr5ISgJmzBB+q2sdU8kDy48dXoqpNoFQSiR/+EP9UiUMTGokASKcQSw5SOFC4nZHvqa7m5uNhB5P65hyD6zPB7S0AMEgtz0Y5H7nxUlu1iXmh5ESyXXrgPXr9amiNzCpkQSIcA58DpLaBytcSGbO5MLhoUR7oDo7Oaf1nXdGd8LKPbDRZhNSglJcLG7WAZEiWVPDCZpeDmMjkxpNiMZJQmF4whKUhtfDX8f/pKTIh+Pljh8t9M+/f+VKbiHBaCkDzz8vTC9oaTFuSenwc4qCkuebBIhITJQ8UGIPvdLcGqnja80tklrt1OuNzDOKlgcklhNlAJYK0J133smmT58u+xoSIMK2nD3L2HXXGbOEsobZhKgYulzcT+g2t1v+XA1ejjkUyxIRd+/ejdbWViMOTRDGw4fR//Mf6dfE4oRV4ss6fhyYM4frBz1nDnDLLUI/jMvVKy+hdHX1huDFztVmfYIMEaBnnnkGjz/+uBGHJgjj4R9SPlIVTkqKsZXlx49zwrNhA/Dll9y/o0cD77/POZkzM7le0WKkpgKDBkk7jG3WJ0j3TOitW7fiu9/9Li699FLR/X6/H36/HwDQ3Nys9/AEETtSyX1DhwITJwL5+cZVlgcCwPTpkeIXDAKPPQb89a/c+TU2Rr7X7ebOsaaGi5SJLcssViBrYZ8gTQIUCARQWFgYsb2srAzvvPMO1q9fL2mClZSUoKSkBADXtJogYkbvKnmph3TiROmZhx7wpt+BA+L7//Uv6fNLSuLSC9atk2+ZUlrKlVGEV8tb1SdIT6fTN998w8aNG8emT5/OpkyZwi655BL2/PPPx+SkIghZjHCqih3T4+mNLhnluJWLugGM9evHjalXZwA1TnANWBoFO3z4MEXBCONRklejhdCHNNYWF0rD3lKhdv6nb19hqxATRCQWLG3HMXz4cLz//vtGHZ6wO2Y1YzfKqer1cmZJTg5n+si1uJC71mh9oUPfGwyK16zxXLjQO6bWrHCbQe04CP0xsH1DBEY5VcOvIZzQ+iy5a5ULe5eWCt+bktIrQnJjxhFUC0boj5m5JkbVKYVfQyihY0S71miFqaHvPX+ec3LPmMHl/lxySW/NmtXOYoMgASL0x8xcE6NW6pAKxQ8cCDz9dO8Y0a5VbWHq+fNcVf4773DLND/3XFyvQEICROiP2WtSGeEPkeoh1NYG/OlP8q8LvVa5GVq098aJn0cOEiBCfxyyJpUsxcVcC45wOjuFJla0a5WbocXDfYoRckIT+sM/dD6feDauFoxcv0uMqirOHyNWjhHaFzr0Wqure4PmvJPZ65VODFRyn/S6brXHMet+m5AOIAnlARGKMLGCuwe5nJzkZK4Pj9HnqNcx1R5Hp3FpWR4iPrCiglvKBwRws6KXXxbm+8ido9acKL2uW+1xTLzfCSlAgY4AKnZUYN5f5qFiRwUCHQYlycUbZiUXhmNFBXe4fyYUxoR9nOXOcds2+UREOfS6brXHkXr9Cy/o/ndPOAEKdASQuToTSz9ciqq9VVj64VJkrs4kEYpGtIxeI1HbqF0PQp3Hl18euT/8AZY6R0D5bCL8WrKy9Ikmqo1KSs3+mpr0/7urMup0xgof0Mq6lSz116kMT6HnJ/XXqWxlnflL1zoKo2qulCDlkzCq/3E4Svs4i53L7NnROxSuXMnYrFnCZZtTUxm76iruR+r6lNaYxeoDUtuO9lvIByRC7dFatAeF08v2YDs+abKmIZNjsLKRlVQou6rKHF+FknC51Dnm58vP3vhZ5caNwNdfC6+lpQVYuFA8hK9mRqo2WVPt7C8WokqUgdAMyEGYOAM6e+EsW1m3khVXFbOVdSvZ2Qsqq8e19mqWPSmN1edqV8hQei1m/T1iGIdmQCKUZpZi6MVDkerhvpVSPakYevFQlGY6I/nLMge6SUlzqnx0ZmZc67UmWejsY+tW8XIPHrlrMWtGavDfPeESEb19vGi8vxG+Rh8+afoEOZfnoDSzFN4+9k9z5x/OY98cQ3uwHZv2bcKL/3oRjfc3Gn/+Xi8CO2rgW/NT1B7/N3Ivux6lZS/Bq3Nymq/R13N9AGceH/vmGHyNPjwwMSyRz27d/aQQS0QMBLjWqVIkJQH9+nEZ2WKY1VrViKTSEBJuBgRwIvTAxAewfu56PDDxAUeIDyD/cBpNoCOATH8elnZuQdWAJizt3IJMf57uMzBVPjqjClHNwOfj6srCcbm43s7d3dz+vDxxv46ZZRwG1qQlpAA5FSsd6GaJX+4VuT3mMU+qJxU5l0t8szu1YFOq2j4piVtaB5B3qjtZfEMgAbIBSv06qh9OHTFL/Jzuo1OMmP8qVHx45Pw6ThXfEEiALEaN09XKh9Ms8eN9dOWF5Zg3dh7KC8vN8XGZDW9C8Q3HkpI4QeJ/5wn161iViW4gLsbCl1Y0j6KiImzevNmq4W1BxY4KLP1wqWB2kepJRXlheaTTFZxgWeFAD3eA8+IXl+JgFsePAxkZwOnT3MyHb8nq8XCNyXi/Dr8GWGj7Vo+He/2yZcCiRbac/Sh5vhMuCmY35EwbMQHiHehi+4wkNHq47ci2nu2+Rp9jooiimN3mI3Tcn/60V3yAXtGZMYPrihgacaqoECZdBoPcMZ54gmuQ5kD/D0ACZDm5V+Ri075NETMgM/w6avH28aI0sxQv/uvFnpnQlgNbzEsF0Bu9mucrFTH+ddu2cb2DTp3iol2htLdzrVirq4XHkHJahzZIE1uI0Obo7gPq6urCI488gsLCQhQUFOCA1CqPBADnOV2tTAXQHT3aTigtiQh93YYNXNlFuPjw7N4deQy59iAWru0eK7oL0J/+9CeMGTMGH374IbZu3YqRI0fqPURc4TSnqy1q6fRyxuqRTaxUxORW2QgnGIw8Bu+0Flsa2sHL9eguQH/5y19w6NAhTJ06FQ8++CCCYi0tCQFOSoy0MhUAgL5tQWIt5QgEgLVrlYmYlAklRfgx+LyfZ5/l/p+c3Hu+dsz+VojuAnTs2DEMGzYMH3/8MQDgjTfeEOz3+/0oKipCUVERmpub9R7edBKtuZnlJqOe3fpiySbmhXDPnsh9YiImZ0J5PFwGdLRjeL3Ao49yVfIvvODoBEQeTWH4QCCAwsLCiO1lZWV49dVX8dZbb/UszfyPf/wDv/vd70SP4/QwvNWhaT4kX3u0FrlX5Joakreslm7ePG7mI7Z9/Xr1x+Mdw0rqnEKdzcEgsGVL5KzG4wGGD+8NnfOvz8ri2ri2tPSG0ZOSgNGjgf/5H24mxe8LDb87VFgAA8PwXq8XdXV1ovsOHjyI+vp6DB8+HPX19XHtA1JVOKkzVhamWpUKAED/IkypFSvCCY+YJSWJO5EzMrgIFgCMGwccPcqJ1YYNwLBhwOOPc4sNXrgAdHQA+/cD69ZxIlVVZUjBp53R3QR77LHH8MYbb6CgoAC7du1CqUNtUyXYtTYrrs1Cq9bSCjf9xMQnNRVYsIATjspKLpzO+0CDQeCLL4B//5vL++HXHONNyKoqx5dVaEH3PKC0tDT87W9/0/uwtsTKHB4p8av+olqQp2Nqyw4zMLg9BADxvB4pJ7LbzQlKuBC++SbXvisUxrjZkZTT2oF5PLFCtWAxYMfaLMaYbJ6O42dHoeJglPiIRdnEGsSnpAAzZ6pzBvfrZ+6y1TaHMqFjwMrmZuEZybz4MTBJs7A0s9S6hmZ6oFfmshxSUTaAm+GENz9bt0587NtvB3btEs6CXC6ux/O6dZHHKS7mcprMLgmxGJoBxYhVOTxSCYwFwwsk83Qcn8VsxoJ5UsmJDQ3q+u8sWgSMGNGbr5OczP3+4IORx6mp4RqPWbHkkcWQADkYMfGTMwttkcUcC2b0QZZLTpTrvxOenQ1wMyA+X+eFF7jfQ9eK54+jZnWPOGvJQSZYnCFnFqpxmluVYySLGX2QtfSZljMNlYT45YS1tFQ6l8gIE9RsYly0IyasWJYnkTl74SwbWTGyZ1mi1F+nspEVIyOWvVH6OtNRu8BeLOOoWYIn1iVypN7//PPC6/V4GHO5rFkcUgO0LA8hQGnhqxW+IkXRObP6IKttdRqraSiV2wRE9gAKD+07uBIeIBMs4VCSxay0SZpeZpqqrG6lmctmEqtpKJXbdM890QtYHR7CpxkQEYGSindVCwhGwRbRuVicu3pkZ4vNusQc4i5X3FTCAyRAhAhKEiz1FA3Lo3OxtvgwyjQUE7YRI4Df/CYuKuEBMsEsQe8Ik9bjhb+veGwxqvZUofZoLe4bfx/gAhqaG0QTLNX2spbD8ra0cvlFSk09I0xDpWUnVvW11gFaFcNk9G7hofV4Yu/r7O6Ex+XB+a7zUY+jdjUPI65BE2IP6z336Nviw0zCUwBs1MpDyfNNJpjJiJkuR04fQeW/K6O+VyxSpNUUEntfsDuI813nFR1Hzzo409rSqqnzcopz14zscAMhE8xkxEyXIAvimepnsOj6RZIPnVSkKGNIhiZTSOw8wgk9TqAjgMr6Sry5+02AAbdn3I6au2tQtadKlzo4U3oMqa3zcoJzVy4FwE6RQglIgEwm94pcbPh/GxBkwl7ZF4IXZBuZSc10vj/k+0j1pKr2n4j5XcLxuDzIuTwHgY4Axr08DodPHwYDZ7HvOrELLze8jF0Ld1nTmEwL0eq8jGzxYRRmZIcbCJlgJlOaWYqU5JSI7Z3dnbJRHymnr8vl0mQKhZtQYqQkp6A0sxS+Rh+azjT1iA8AMDAcbT3qnEJWQHudl1KsqNOyqkGbTpAAmYy3jxfL8pchOSlZsD3arEWu/0/GkAzMGDkDc66do9h/Eup3ybw0M+J8kpOSsSx/Gbx9vKg9WovO7s6IYwRZ0DmFrICxD6ueq3WowazscIMgAbKARVmLcOXAK1XNWsScvp3dndi8fzM2/ncjthzYgl0ndqF4bDF8jT5FDcd4v0t1aXXE+Vw58EosyloEgBO/cIECek00S1Ez6zDyYY3mDDZydqTH7M0iKAyvAT3yeLSsLBH6ns4uTnz4qBXACUe/5H5o62xTHc6WOx8xH5ALLoxIG4FdC3cpPrbulfV2CkHLrdaxZo19ztNElDzfJEAqsXopHp55f5mHqr2RH/gkVxK6WW/D9Fhyc8KTFF//7HW8tfstgAE/yviRbNQu9DiG3a+KCs7UCXfAlpebHwGSOxfAPudpIpQHZAC2qFuCuE/I7XILxAfQVtIgVueVtzYPi7IWYef9O7Fz4U48mvuoIgEx9H6Z0aBMKXL+JTudp80gAVKJ5XVL3yLmE0pLTUOKWxhh01LS4Jg6r1iXVtYTOf+Snc7TZpAAqcTstdGl+uSIZQ/vXrQbwwYMizk7WU/RMPR+lZYicGU6KnI9mDcXqMj1IHBlunUhaClnsNboW5y1XxVDdx9QfX09fvazn6FPnz647LLL4Pf7kZwcGUEBnOkDOh44joxVGTh1/hS6WTdS3CkYNmCYIT4gLf4TPZZNdkqdV6AjgMzKcTjW2oR2dCIVyRg68HI0LoruGDcdNUtA8693uOPaEh/QihUrsGLFClRXV2Pw4MHYsmWL3kNYRqAjgLy1eWjrbEM364bb5Ya3rxc1d9cY8oHXYgrpsUqHmjqvaJ0Mjazz8jX6cCzQgnZwOUrt6MSxQIs9kyPVhsodXuOlFN0FaOzYsWhtbQUAnDlzBpdcconeQ1hGuCB0sS60dbShao9I+FUHrPI3KRUNpU3JjFq6yC7+OENIEMe17gL0wx/+EIsXL8aYMWPQ2tqKSZMmCfb7/X4UFRWhqKgIzc3Neg9vKEZ94KVmEWb7m0JRIhpWRwStvD+GrzCbII5rTT6gQCCAwsLCiO1lZWXw+XyorKzEuHHjsHz5cqSmpuLBBx8UPY7TfEB6+kZ45HwkAHTznxiRDCiVizRv7Dysn2t8Hx2rcrJMGTdBfECaquG9Xi/q6upE9/l8vh6za8iQIThy5IiWIWyJ1HLIsawFLzeLeGDiA7os/ayq6bsKsr6TFVHZb2YnQ6uWxo72N9MFpd0QHY7u7TiWL1+OuXPnom/fvnC73fD7/XoPYRlGfOCjtTbV0icnfLZzIXgh6gPDv2fbkW0AuGr3guEFktcX6Ajg5YaX0cW6era54EJ6//SYBFktpvQRCkPPdrSy2HEFEJ3RXYDy8/Oxfft2vQ9rG/T+wCvph6zGfBKb7bhdbtkHJvw9PFv2b5GcKfkafWg52yJo0eFJ8mBh1kL7hcB1xvIe1nEEJSKGocW5GItDMlrIW+3yN2LmwYWuC/C4hN81oQ9M+Ht4znedl3Qqi80COrs70dDcoPjanYqe7WgTHeqIGAJf9d10pgmd3Z3Y+N+NWLljpWzFd6z+lWhmnVp/g5QwePt40dXdJeq7kmvPKmVaxDoLsOXa8wqxyvcUj9AMKITK+kocPn24p/lWZ3cnDp8+jMp66YbxeoSipULegY4A1jauVRX6lwpNL8tfJpnXI/ae0PeKiUosswA9FzW0CqNymxINEqAQ3tz9psCnAXDO2Ld2vyX5HiNzgzJXZ2LP/+2J2Cc30wgXhuSkZLhdboABxWOLkXN5DmqP1sLX6Ot54KXas6a4UyRFJZYMZ6vzhwj7QCZYKFIZUTKZUtFMEa2mBv+QhrdC9bg8sjMNXhgq6yvxzLZncL7zPAKdASz7eBke/9/HkZyULGoq8iZF9RfV4FPD8ofny56vVoe8aVEkwvaQAIVwe8bt2HVil2AW5IILP8r4keR75HKDYvEPSfllMi7NQPXd1bLv9/bxoq+7L7q6u3pydPjOicFu7vdwX5KZ4WyKIhE8ZIKFsOj6RRiRNqInYuRxeTAibQQWXb9I8j1ypkgspoaUL2fBdQsUzaDUrPtlNhRFIngcMQMyK2Li7ePFroW7VEc3pGYPW49s1WxqxJp1rWTdL6tmHRRFInhs3xPaLj2Y1RLoCGDEyhH4+tzXgu0p7hQ8d+NzikydWHr78PftyzNfChrX86hZQ96p4XLCWgyrBTMTU+puDMDX6ENbR1vEdm8fr+JZTCx+GX6WcdfGu/D3z/8uKJlIciVhxsgZWDdrnSDkHy40AAypISMIHtv7gJza80XKBzP04qGGjCeWje3t44UnySMQHwDoZt1IdicLxEcsL6eyvpLC5YSh2F6ArOz5ooZwAcgamiWa3Lf3//bqnnQnJSDHA8cR7A4iySX8M4ffP7FZ5hetX+DZ6mc1i7/h/XKIuMD2JpgRLTD0Rizcnt4/Hen903G09aigXUVnd6fAhNTDxyJlpmasyuhpH8sjllwoVb7ReqE1Yiy3y43Ors6eGZbS+0GmGyGG7WdARvYUjobSb3ExAWg524KFExbie5d+L+L1/CxCbuaiZvYgZaaeOn9KsN3tcqNoVFHE/ZMrxQini3Vh8/7NqgtiyXQjxLC9AAHW1N2oqVeSEoCGlgbcfd3dkiak3MxFTZ2U0kUKu1iXwPfDI1WKIYVclTzgXL8dYT6OECArUPMtLuenkku6UzJzUTJ7EBsjvKYN4MwvMd9Z6Cwz87LMiNYd4T4k/rzUFsTazW9HWA8JkARqvsXDBSDFnYJ+yf2w7cg2+Bp9qLm7RtSEjGV55VDzMHyMG6+6EUkif1q5FAB+lll9dzWGpw0XiNmglEGqBEVOdMk5TYRi+0REq1DbgD60pWn10Wq0dbRFTZzk+w/xjmqPy4MBKQPQ1tEmSB4MHzdaI/v0F9IR6Ix8sOdcOwfvFL8T9drDEyCLxxYjb21ezAskAvo12SfsT1wkIlqF2ugbP4MAgC0HtqhKnHS5XADj/u3ftz8GpAxAy9kWyXGjmYcXui5EjOFxeTDlyimKom5iCZDhpRPFY4tljyN2jIodFY5MKiWMgwRIAq31SkpbTQQ6Arhr4134ovWLnkTBzu5OnAicwNNTn0Zfd1/JceXGYIxFtPAAgL6evigeW6w5PB4qKFrD7NSGgwiHfEBhhPtWSjNLVUXflDhg+Qf4b5//LSJLuT3YjobmBtmon9wYUiH1FE8KXv/sdV3C41rD7OScJsIhAQpBj1ahSlpN8A9wuLOZf320B1JujNLMUvTr0y/iPW0dbXhz95u6hMe1htmpDQcRDplgIehR+KrEdJOqE3O73IoeyGhj5F2Rh43/3Sh4z/mu83AxF1I9qTE3AtO6IKGebTioSj8+0DwDWr16NUaNGoXRo0cLtr/33nuYNGkSsrOz8dFHH8V8gmai9ptdKqQcLXFSzBRJciVh5jUzVa2mITVG9rBsuOASvN4FF2ZfOzvmGcjxwHE8V/ucQHzULEioR1JpPDS1Jzg0C9CsWbOwd+9ewbauri48/vjj+OCDD/Dee+/h0UcfhYVRftVE81GECs6KT1Zg3MvjND0EYqbIVWlXCdpjxIRLfHPf5L4xlbUEOgLIWJWBk+0nBdvNXpCQSj3iB80m2JAhQyK27du3DyNHjsSAAQMAAOnp6WhqasIVV1yh/QxNRCz0nt4/HRe6LmDO23NQ/UU1Ah0BnO86j43/3Yhgd7An41iNuWZ0R8D6Y/Wiq3s0NDfE1GPI1+jDqfOnIrabvSAhRdPiB119QKdPn0ZaWlrP7wMHDsTJkycFAuT3+3vWi29ubtZzeM2E+hPum3AfwICGlgZMSJ+AlxtexpMfPylaLR6OmofAyCbwRjV9rz1aK+o4d7vcpkayqKl9/CArQIFAAIWFhRHby8rKUFZWFrE9LS0Nra2tPb+fOXMGgwYNErympKQEJSUlALhMSasRy2nhs3P59c+jNXfnsctDYFQLk9wrcvHu5+9GtHhNS00zNZLlhBYthDJkBcjr9aKurk7xwUaNGoUDBw7g7Nmz6OrqQktLi+3NLzl/QrSVJVxwwe1yI8iCtnoIjDLx+Aef7zOd5ErCoJRB2L1ot6kRKGpqHz9oNsE2bNiAVatWoampCYWFhSgvL0dWVhZ+9atf4aabbgIAlJeXc2UGNkbOnyC3sgTvH1o4YSEaWhps9xAoMfHUhrLt9ODHZMIGAoDPB9TWArm5QGkp4LXH3y3RSPhiVLmi09LM0ojiyX59+mHKFVOirhpqd5y62kjMBAJAZiZw7BjQ3g6kpgJDhwKNjSRCOqPk+U74TGi57FyxboyHlxzGX+f91bTGaEaRsKFsn69XfADu32PHuO2E6SR8JnQ0s8LMJYv1QolplbCh7NraXvHhaW8HPvkEeCCOr9umJLwAAc4UGSmUVqonbCg7NxfYtEkoQqmpQE6cX7dNSXgTLN5QalqpKQyNqy6GpaWczyf124x33gdUan30MhGhGZBCnFL8qNS0UhrRirsldrxezuHs83FmV04ORcEshARIAU56CNWYVkpMT6cujS2L18v5e8jnYzlkginASREjrT13pMwsWmKHMBKaASnASREjLcmCcjM8o53VTjFtCWMgAVKA3g+h0Q+d2qie3AzPyLorJ5m2hDGQCaYAPVuJmtFMS23USm6GZ+TS2E4ybQljoBmQAvSsgdLq1FU6a9Iyq4g2wzMqT8pJpi1hDDQDUohe69NrceqqmTVpmVVY1SyeVskgSIBMRstDp0ZUtAickWaWHLRKBkECZDJKH7pQP87axrWKRUXrrIJfNz7n8hzUHq2Fr9FneMazVcJH2IeEb8dhBWLrpoc+dOF+nOSkZEH/aYATlacLnkZfT1+BXwjQtv56wrbnIAxDyfNNAmRDxHoUhXdfTO+fDgARa8g33t8IAKod5nJ9kcghTGiB+gE5FDE/DgPDpd5LMXv0bJQXlmNh1kJBv+pQv5AWhzllPBNWQAJkQ6TWdz8ROIHPvvoMpZmlqD9Wr6tgUESKsAISIBvCO6qTk5IF24MsiGPfHEPlvysR7A4iySX888UiGBSRIqyAEhFtCB8dyvPl4dMTnwr2tQfb8Uz1M+jq7hKs0ZXiTsHQi4eieGwxKnZUqC7zsFPDeSJxICe0jRFzDHtcHrhcLsHCiG6XGzOvmYmXbnkJeWvzKJJF2AJyQjscMbMoJTklYlXWLtaFZHcyqvZUUW0V4SjIBLMxYmbRha4LEUtF874fqq0inIbmGdDq1asxatQojB49umfbiRMnMHXqVOTl5WHy5Mmor6/X5SQTmfCQ+qKsRZLOYopkEU5DswDNmjULe/fuFWy76KKLsH79etTU1GDNmjV49NFHYz5BQohc+QJFsginodkEGzJkSMS2/v37o3///gCAPn36wOMhC88IpNpjUCSLcBqGKER3dzcefPBB/OIXv4jY5/f74ff7AQDNzc1GDJ/QxNMaZ0T8IytAgUAAhYWFEdvLyspQVlYm+b6f/OQnuPHGGzFt2rSIfSUlJSgpKQHAhekIgkhcZAXI6/Wirq5O1QGfeOIJDBw4EIsXL47pxIj4gJrOE3JoNsE2bNiAVatWoampCYWFhSgvL0dKSgrKy8sxefJkFBQU4LLLLsP69ev1PF8iBswWA2o6T0RDswDNnj0bs2fPjtje1dUV0wkRxmCFGMTlooaErlAmdIJgxQoU1OKDiAYJUIJghRhQYiQRDRKgBMEKMaDESCIaJEAJghViQE3niWhQqnKCYFWWNCVGEnKQACUQJAaE3SATjCAIy6AZkA2h7GEiUSABshmUPUwkEmSC2QwrEgYJwipIgGwGZQ8TiQQJkM2g7GEikSABshmUPUwkEuSEthnUVpVIJEiAbAglDBKJAplgBEFYBgkQQRCWQQJEEIRlkAARBGEZ5IQ2GarzIoheSIBMhOq8CEIImWAmQnVeBCGEBMhEqM6LIIRoFqDVq1dj1KhRGD16dMS+U6dOIS0tjRYlDIPqvAhCiGYBmjVrFvbu3Su6b/ny5cjNzdV8UvEK1XkRhBDNTughQ4aIbj969ChaWlqQlZWl+aTiFarzIgghukfBnn76aTzxxBN4++23Rff7/X74/X4AQHNzs97D2x6q8yKIXmQFKBAIoLCwMGJ7WVkZysrKIrbv3r0bLpcL1157reQxS0pKUFJSAgAoKipSe74EQcQRsgLk9XpRV1en+GANDQ34/PPPcfPNN+PAgQPwer0YPXo0rrvuuljPkyCIOESzE3rDhg0oLCxEU1MTCgsLUV9fjwULFqCmpgbvv/8+7rjjDixdupTEhyAISTT7gGbPno3Zs2dL7n/qqae0HpogiASBEhEJgrAMEiCCICzD0mLUgwcPqo6ENTc34zvf+Y5BZ+SMc7B6fDoHe4xvh3OQG//gwYPRD8AcxowZM6w+BcvPwerx6RzsMb4dziHW8ckEIwjCMhwnQHwSYyKfg9Xj0znYY3w7nEOs47sYY0yncyEIglCF42ZABEHEDyRABEFYhiME6P7770dBQQEKCgowePBgvPvuu4L9R44cwaBBg3pes337dt3P4amnnsKYMWNQUFCAmTNnir7mmWeeQW5uLm644QZ8+eWXuo7/2GOPITs7G5MmTUJ5eXnEfiPvwZo1a5CdnY3Jkyfjs88+E+w7dOgQCgoKkJubixUrVug2Js+ePXswefJkTJkyBdOmTcOhQ4cE+xcsWIDx48ejoKAA999/v+7j81x00UU993bjxo2CfUbfg+3bt/eMPWHCBIwfP16w36h7cO7cOWRnZ2PgwIE9zQXb29sxf/585OXl4c4770RHR0fE++Q+LxHoEoszic7OTnb11Vez9vZ2wfbDhw+z6dOnGzr2k08+yd566y3J/Xv27GE33XQTY4yxjz76iN111126jr9v3z7GGGNdXV0sOzubHT58WLDfqHtw8uRJNn78eNbR0cH279/Ppk2bJtg/d+5cVltby7q7u1lBQQE7dOiQruOfOHGCtba2MsYY27JlC1uwYIFg/1133cW2b9+u65hiXHPNNZL7jL4Hofz+979nzz77rGCbUfcgGAyylpYWwWf/j3/8I/vNb37DGGPsl7/8JVu7dq3gPdE+L+E4YgbE88EHHyA/Px8pKSkR+xobG5GXl4f77rsPbW1thoxfXl6OyZMn489//nPEvm3btuEHP/gBAGDq1KnYuXOnrmOPGjUKAJCUlASPxwOPJzKH1Ih7sGPHDhQUFCA5ORkjR47E119/je7u7p79e/fuRU5ODlwuF4qKilBTU6PLuDxDhgzBgAEDAADJycmi171kyRLk5+fj/fff13XsUJqbm5Gfn4/58+fjq6++Euwz+h6E8uabb+L222+P2G7EPXC73bjssssE26qrq3HrrbcCAG699VZUV1cL9kf7vITjKAHy+/2iYb/09HQcPHgQNTU1GD58uKiJEiuLFy/Gp59+is2bN+Oll17Cvn37BPtPnz6NtLQ0AIDL5UIwGNT9HADg7bffxlVXXYVhw4YJtht1D0KvC+BatJw5c6bn99AP18CBA3Hy5Eldxg2nvb0dTz75JB566CHB9t/+9rfYsWMHqqqq8NBDD6G1tdWQ8Q8fPoxt27bhtttuw8MPPyzYZ9Y92LdvH1JSUnDllVcKtpt1DwDh50HsWqN9XsKxjQAFAgFMmjQp4mfNmjUAgLa2NtTX16OgoCDivX379oXXy7U1nT9/vubZh9w5DB48GABw8cUXY/r06di1a5fgvWlpaYI/vNg3dSzjA8DWrVvxyiuvoLKyMuK9et2DcMKvKxAI9MxIAG5GxnPmzBkMGjRIl3FDCQaDmD9/Ph555JGIZnf83+XSSy9FVlYW9u/fr/v4oePMmzcPn376qWCfGfcAkP4CNuseAMLPg9i1Rvu8RKC74WgQr7/+Onv44YdF9505c6bn/6+++ipbsmSJ7uPzfohgMMimTZvGGhsbBfv/85//9KSlb9u2TXcfUH19PZs4cSI7efKk6H6j7sHJkydZVlYW6+zsZIcOHWJTp04V7J87dy7bsWMHY4yxG264gR08eFCXcXm6u7vZHXfcwVatWiW6n/+7nDt3jmVkZLATJ07oOj5jjAUCARYMBhljnH9v5syZgv1G3wOeMWPGiP79jb4H4T6g5557jjHG2NNPPy3qA5L7vITjGAG6+eab2c6dOwXbfvzjHzPGGNu0aRMbP348y8vLY7fccgv76quvdB//nnvuYdnZ2ez6669ny5cvjzgHxrg/VE5ODps2bRr74osvdB1/woQJ7Nprr2X5+fksPz+/RwDNuAerV69mkyZNYrm5uezTTz9lW7ZsYX6/nzHG2P79+1l+fj7Lyclh5eXluo3Js2nTJpaamtpz3UuWLBGMP2PGDJaTk8Ouv/569tprr+k+PmOc+GdmZrK8vDw2bdo0tn//flPvAWOM1dXVsdtuu63nd7PuwcyZM9mIESPY9773PbZ48WLW1tbGiouL2eTJk1lJSQm7cOECY4yxJUuW9Ihj+OdFDsqEJgjCMmzjAyIIIvEgASIIwjJIgAiCsAwSIIIgLIMEiCAIyyABIgjCMkiACIKwjP8PlBzWLiWSeoAAAAAASUVORK5CYII=\n"
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x,y = make_blobs(n_samples=200,\n",
    "                 n_features=2,\n",
    "                 centers=2,\n",
    "                 cluster_std=2,\n",
    "                 random_state=40)\n",
    "y_ = y.copy()\n",
    "# 将标签转换为1/-1\n",
    "y_[y_==0]=-1\n",
    "y_.astype(float)\n",
    "\n",
    "def draw_fig(x,y,w,b):\n",
    "    plt.figure(dpi=64,figsize=(5,5))\n",
    "\n",
    "    # 正负实例的散点图，这儿使用了一个Bool索引，切片混合索引\n",
    "    plt.scatter(x[y==1][:,0],x[y==1][:,1],color='r')\n",
    "    plt.scatter(x[y==-1][:,0],x[y==-1][:,1],color='g')\n",
    "\n",
    "    # 画分离超平面\n",
    "    if w and b:\n",
    "        x1 = np.arange(-1, 4, 0.1)\n",
    "        x2 = (w[0] * x1 + b) / (-w[1])\n",
    "        plt.plot(x1,x2)\n",
    "\n",
    "    plt.show()\n",
    "\n",
    "print(x.shape)\n",
    "print(y.shape)\n",
    "draw_fig(x,y_,None,None)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "outputs": [],
   "source": [
    "# 划分训练集和测试集\n",
    "x_train,x_test,y_train,y_test = train_test_split(x,y_,test_size=0.2,random_state=1)"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "markdown",
   "source": [
    "#### 决策树桩 decision stump\n",
    "\n",
    "单层决策树，通过给定阈值进行分类\n",
    "\n",
    "决策树桩根据一个属性的单个判断来确定最终的分类结果\n",
    "\n",
    "做的事：\n",
    "\n",
    "1. 从所有的属性中，选择哪一个属性作为决策树桩（弱学习器）\n",
    "2. 设置该决策树桩的阈值\n",
    "3. 大于/小于阈值的时候预测为正"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "outputs": [],
   "source": [
    "# 单层决策树（decision stump，也称决策树桩）\n",
    "class decision_stump(object):\n",
    "    def __init__(self):\n",
    "         # 基于划分阈值决定样本分类为1还是-1\n",
    "         self.label=1\n",
    "         # 特征索引\n",
    "         self.feature_index=None\n",
    "         # 特征划分阈值\n",
    "         self.threshold=None\n",
    "         # 指示分类准确率\n",
    "         self.alpha=None"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "outputs": [],
   "source": [
    "# 定义Adaboost算法\n",
    "class Adaboost(object):\n",
    "    # 弱分类器个数\n",
    "    def __init__(self,n_estimators=10):\n",
    "        self.estimators = None\n",
    "        self.n_estimators = n_estimators\n",
    "\n",
    "    def fit(self,x,y):\n",
    "        m,n = x.shape\n",
    "        # 1.初始化训练集权值分布为均匀分布 1/N\n",
    "        w = np.full(m,(1/m))\n",
    "        # 初始化基础分类器列表\n",
    "        self.estimators = []\n",
    "        # 2. for m in （1，2，3，...,m)\n",
    "        for ixe in range(self.n_estimators):\n",
    "            # 2.1 训练一个弱分类器：决策树桩\n",
    "            estimator = decision_stump()\n",
    "            min_error = float('inf')\n",
    "\n",
    "            \"\"\"\n",
    "                训练弱分类器\n",
    "            \"\"\"\n",
    "            # 遍历数据集特征，根据最小分类误差率选择最优划分特征\n",
    "            # 遍历n个特征\n",
    "            for i in range(n):\n",
    "                # 获取特征值\n",
    "                values = np.expand_dims(x[:,i],axis=1)\n",
    "                # 特征值去重\n",
    "                unique_values = np.unique(values)\n",
    "                # 尝试将每个特征值作为分类阈值\n",
    "                for threshold in unique_values:\n",
    "                    p = 1\n",
    "                    # 先初始化所有的预测值为1\n",
    "                    pred = np.ones(np.shape(y))\n",
    "                    # 小于分类阈值的预测值为-1\n",
    "                    pred[x[:,i]<threshold]= -1\n",
    "                    # 2.2 计算误差率\n",
    "                    error = sum(w[y!=pred])\n",
    "                    # 若分类误差>0.5则进行正负预测值翻转\n",
    "                    if error>0.5:\n",
    "                        error = 1 - error\n",
    "                        p = -1\n",
    "\n",
    "                    # 得到最小误差则保存相关参数配置\n",
    "                    if error<min_error:\n",
    "                        estimator.label = p\n",
    "                        estimator.threshold = threshold\n",
    "                        estimator.feature_index = i\n",
    "                        min_error = error\n",
    "\n",
    "            # 2.3 计算基础分类器的权重\n",
    "            estimator.alpha = 0.5 * np.log((1-min_error)/(min_error+1e-9))\n",
    "            # 初始化所有预测值为1\n",
    "            preds = np.ones(np.shape(y))\n",
    "            # 获取所有小于阈值的负类索引\n",
    "            negative_idx = (estimator.label*x[:,estimator.feature_index] < estimator.label * estimator.threshold)\n",
    "            # 负类设置为 -1\n",
    "            preds[negative_idx]=-1\n",
    "            # 更新样本权重\n",
    "            w *= np.exp(-estimator.alpha*y*preds)\n",
    "            w /= np.sum(w)\n",
    "            # 保存弱分类器\n",
    "            self.estimators.append(estimator)\n",
    "\n",
    "            print('decision stump {}, param: label {}; threshold {:.2f}; feature_ix {}; alpha {:.2f}'.format(ixe,estimator.label,estimator.threshold,estimator.feature_index,estimator.alpha))\n",
    "\n",
    "    def predict(self,x):\n",
    "        m = len(x)\n",
    "        y_pred = np.zeros((m,1))\n",
    "        for estimator in self.estimators:\n",
    "            predictions = np.ones(np.shape(y_pred))\n",
    "            negative_idx = (estimator.label * x[:,estimator.feature_index] < estimator.label * estimator.threshold)\n",
    "            predictions[negative_idx]=-1\n",
    "            #  对每个弱分类器的预测结果进行加权\n",
    "            y_pred += estimator.alpha*predictions\n",
    "        # 返回最终的预测结果\n",
    "        y_pred = np.sign(y_pred).flatten()\n",
    "        return y_pred"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "decision stump 0, param: label 1; threshold 2.32; feature_ix 0; alpha 1.62\n",
      "decision stump 1, param: label 1; threshold -7.02; feature_ix 1; alpha 1.52\n",
      "decision stump 2, param: label 1; threshold 2.77; feature_ix 0; alpha 0.75\n",
      "decision stump 3, param: label 1; threshold -5.04; feature_ix 1; alpha 0.82\n",
      "decision stump 4, param: label 1; threshold -0.05; feature_ix 0; alpha 0.93\n",
      "accuracy:1.0\n"
     ]
    }
   ],
   "source": [
    "clf = Adaboost(n_estimators=5)\n",
    "clf.fit(x_train,y_train)\n",
    "\n",
    "y_pred = clf.predict(x_test)\n",
    "acc = accuracy_score(y_test,y_pred)\n",
    "print('accuracy:{}'.format(acc))"
   ],
   "metadata": {
    "collapsed": false
   }
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "outputs": [],
   "source": [],
   "metadata": {
    "collapsed": false
   }
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
